Neural network (NN) designed for challenging machine learning tasks is in general a highly nonlinear mapping that contains massive variational parameters. High complexity of NN, if unbounded or unconstrained, might unpredictably cause severe issues including over-fitting, loss of generalization power, and unbearable cost of hardware. In this work, we propose a general compression scheme that significantly reduces the variational parameters of NN by encoding them to multi-layer tensor networks (TN's) that contain exponentially-fewer free parameters. Superior compression performance of our scheme is demonstrated on several widely-recognized NN's (FC-2, LeNet-5, and VGG-16) and datasets (MNIST and CIFAR-10), surpassing the state-of-the-art method based on shallow tensor networks. For instance, about 10 million parameters in the three convolutional layers of VGG-16 are compressed in TN's with just $632$ parameters, while the testing accuracy on CIFAR-10 is surprisingly improved from $81.14\%$ by the original NN to $84.36\%$ after compression. Our work suggests TN as an exceptionally efficient mathematical structure for representing the variational parameters of NN's, which superiorly exploits the compressibility than the simple multi-way arrays.

Complex systems, such as Artificial Intelligence (AI) systems, are comprised of many interrelated components. In order to represent these systems, demonstrating the relations between components is essential. Perhaps because of this, diagrams, as "icons of relation", are a prevalent medium for signifying complex systems. Diagrams used to communicate AI system architectures are currently extremely varied. The diversity in diagrammatic conceptual modelling choices provides an opportunity to gain insight into the aspects which are being prioritised for communication. In this philosophical exploration of AI systems diagrams, we integrate theories of conceptual models, communication theory, and semiotics. We discuss consequences of standardised diagrammatic languages for AI systems, concluding that while we expect engineers implementing systems to benefit from standards, researchers would have a larger benefit from guidelines.

This short paper examines diagrams describing neural network systems in academic conference proceedings. Many aspects of scholarly communication are controlled, particularly with relation to text and formatting, but often diagrams are not centrally curated beyond a peer review. Using a corpus-based approach, we argue that the heterogeneous diagrammatic notations used for neural network systems has implications for signification in this domain. We divide this into (i) what content is being represented and (ii) how relations are encoded. Using a novel structuralist framework, we use a corpus analysis to quantitatively cluster diagrams according to the author's representational choices. This quantitative diagram classification in a heterogeneous domain may provide a foundation for further analysis.

In this article, we propose a multimodal perspective to diagrammatic representations by sketching a description of what may be tentatively termed the diagrammatic mode . We consider diagrammatic representations in the light of contemporary multimodality theory and explicate what enables diagrammatic representations to integrate natural language, various forms of graphics, diagrammatic elements such as arrows, lines and other expressive resources into coherent organisations. We illustrate the proposed approach using two recent diagram corpora and show how a multimodal approach supports the empirical analysis of diagrammatic representations, especially in identifying diagrammatic constituents and describing their interrelations.

We report on the spring 1992 symposium on diagrammatic representations in reasoning and problem solving sponsored by the American Association for Artificial Intelligence. The symposium brought together psychologists, computer scientists, and philosophers to discuss a range of issues covering both externally represented diagrams and mental images and both psychologyand AIrelated issues. In this article, we develop a framework for thinking about the issues that were the focus of the symposium as well as report on the discussions that took place. We anticipate that traditional symbolic representations will increasingly be combined with iconic representations in future AI research and technology and that this symposium is simply the first of many that will be devoted to this topic. The emphasis of this symposium was diagrammatic (or pictorial) representations in problem solving and reasoning.

In earlier work, we have shown how a cognitive architecture can be augmented with a diagrammatic reasoning system to produce a bimodal cognitive architecture. In this paper, we show how this bimodal architecture is also bi-representational (multi-representational in the general case) by describing a desiderata for representational formalisms and showing how the diagrammatic representation in biSoar satisfies these requirements.

We give a brief overview of the main characteristics of diagrammatic reasoning, analyze a case of human reasoning in a mastermind game, and explain why hybrid representation systems (HRS) are particularly attractive and promising for Artificial General Intelligence and Computer Science in general.

We report on the spring 1992 symposium on diagrammatic representations in reasoning and problem solving sponsored by the Association for the Advancement of Artificial Intelligence. The symposium brought together psychologists, computer scientists, and philosophers to discuss a range of issues covering both externally represented diagrams and mental images and both psychology -- and AI-related issues. In this article, we develop a framework for thinking about the issues that were the focus of the symposium as well as report on the discussions that took place. We anticipate that traditional symbolic representations will increasingly be combined with iconic representations in future AI research and technology and that this symposium is simply the first of many that will be devoted to this topic.

We report on the spring 1992 symposium on diagrammatic representations in reasoning and problem solving sponsored by the Association for the Advancement of Artificial Intelligence. The symposium brought together psychologists, computer scientists, and philosophers to discuss a range of issues covering both externally represented diagrams and mental images and both psychology -- and AI-related issues. In this article, we develop a framework for thinking about the issues that were the focus of the symposium as well as report on the discussions that took place. We anticipate that traditional symbolic representations will increasingly be combined with iconic representations in future AI research and technology and that this symposium is simply the first of many that will be devoted to this topic.

We distinguish diagrammatic from sentential paper-and-pencil representationsof information by developing alternative models of information-processing systems that are informationally equivalent and that can be characterized as sentential or diagrammatic. Sentential representations are sequential, like the propositions in a text. Dlogrammotlc representations ore indexed by location in a plane. Dio-grommatic representations also typically display information that is only implicit in sententiol representations and that therefore has to be computed, sometimes at great cost, to make it explicit for use. We then contrast the computational efficiency of these representotions for solving several illustrative problems in mothe-matics and physics. When two representotions are informationally equivolent, their computational efficiency depends on the information-processing operators that act on them. Two sets of operators may differ in their copobilities for recognizing patterns, in the inferences they con carry out directly, and in their control strategies (in portitular. Diogrommotic ond sentential representations sup port operators that differ in all of these respects. Operators working on one representation moy recognize feotures readily or make inferences directly that are difficult to realize in the other representation. Most important, however, are differences in the efficiency of scorch for information and in the explicitness of information. In the representotions we call diagrammatic. Therefore problem solving con proceed through o smooth traversal of the diagram, and may require very little search or computation of elements that hod been implicit. "a picture is worth 10,OOO words" is a Chinese proverb. On inquiry, we find that the Chinese seem not to have heard of it, but the proverb is certainly widely known and widely believed in our culture. To understand why it is advantageous to use diagrams-and when it is-we must find some way to contrast diagrammatic and non-diagrammatic representations in an information-processing system.